Information processing apparatus, information processing method, program, and medical observation system

ABSTRACT

Provided is an medical information processing apparatus including processing circuitry that, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, selects at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2016-119687 filed Jun. 16, 2016, the entire contents ofwhich are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, a program, and a medical observationsystem.

BACKGROUND ART

These days, various medical observation apparatuses such as medicalendoscopes and medical microscopes are becoming used in the medicalfield. In particular, in the medical endoscope, as disclosed in PTL 1below an attempted to, not only use a normal observation image in thevisible light region, but also image an observation site by irradiationwith light in specific wavelength ranges to generate spectral images inthe specific wavelength ranges, and cause the spectral images to bedisplayed on a display screen is made.

CITATION LIST Patent Literature

PTL 1: JP 2012-81087A

SUMMARY Technical Problem

However, in a medical endoscope like that disclosed in PTL 1 above, thechoice of an image that is caused to be displayed on a display screendepends on the decision by the manipulator of the medical endoscope;therefore, there is a case where, although there is a relativelyappropriate image, the manipulator is not aware of this, andconsequently the efficiency of working in endoscopic observation cannotbe improved.

Furthermore, circumstances like the above may occur not only inobservation with a medical endoscope but also in the case of working inobservation with a medical microscope.

Thus, in view of the circumstances mentioned above, an embodiment of thepresent disclosure proposes an information processing apparatus, aninformation processing method, a program, and a medical observationsystem that can recommend a relatively appropriate vital observationimage to the user during working using a medical observation apparatus.

Solution to Problem

According to an embodiment of the present disclosure, there is provideda medical information processing apparatus including: processingcircuitry configured to, based on surgical situation informationconcerning surgical characteristics at a time of observing an interiorof a living body, select at least one of a plurality of biologicalimages each having a different wavelength range or at least one ofsecondary images generated from the plurality of biological images eachhaving a different wavelength range, as a recommended image.

According to an embodiment of the present disclosure, there is provideda medical information processing method including: selecting, based onsurgical situation information concerning surgical characteristics at atime of observing an interior of a living body, at least one of aplurality of biological images each having a different wavelength rangeor at least one of secondary images generated from the plurality ofbiological images each having a different wavelength range, as arecommended image.

According to an embodiment of the present disclosure, there is provideda non-transitory computer readable medium having stored therein aprogram that when executed by a computer causes the computer to executea medical information processing method, including: selecting, based onsurgical situation information concerning surgical characteristics at atime of observing an interior of a living body, at least one of aplurality of biological images each having a different wavelength rangeor at least one of secondary images generated from the plurality ofbiological images each having a different wavelength range, as arecommended image.

According to an embodiment of the present disclosure, there is provideda medical observation system including: a medical observation apparatusincluding first processing circuitry configured to generate a pluralityof biological images by capturing images of an interior and exterior ofa living body in a plurality of wavelength ranges, and an informationprocessing apparatus including second processing circuitry configuredto, based on surgical situation information concerning surgicalcharacteristics at a time of observing the interior of the living body,select at least one of the plurality of biological images havingdifferent wavelength ranges or at least one of secondary imagesgenerated from the plurality of biological images, as a recommendedimage.

Advantageous Effects of Invention

As described above, according to an embodiment of the presentdisclosure, a relatively appropriate vital observation image can berecommended to the user during working using a medical observationapparatus.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an illustration diagram schematically showing an overallconfiguration of a medical observation system according to an embodimentof the present disclosure.

FIG. 2A is an illustration diagram schematically showing an example ofan imaging unit included in the medical observation apparatus accordingto the embodiment.

FIG. 2B is an illustration diagram schematically showing an example ofthe imaging unit included in the medical observation apparatus accordingto the embodiment.

FIG. 2C is an illustration diagram schematically showing an example ofthe imaging unit included in the medical observation apparatus accordingto the embodiment.

FIG. 3 is a block diagram schematically showing an example of aconfiguration of an information processing apparatus according to theembodiment.

FIG. 4 is a block diagram schematically showing an example of aconfiguration of an image recommendation processing unit included in theinformation processing apparatus according to the embodiment.

FIG. 5 is an illustration diagram for describing a situation identifyingdatabase included in the information processing apparatus according tothe embodiment.

FIG. 6 is an illustration diagram showing an example of an absorptionspectrum of main substances in a living tissue.

FIG. 7 is an illustration diagram for describing a recommended imagedatabase included in the information processing apparatus according tothe embodiment.

FIG. 8 is an illustration diagram schematically showing examples of adisplay screen in the information processing apparatus according to theembodiment.

FIG. 9 is an illustration diagram schematically showing examples of thedisplay screen in the information processing apparatus according to theembodiment.

FIG. 10 is a flow chart showing an example of the flow of an informationprocessing method according to the embodiment.

FIG. 11 is a block diagram showing an example of the hardwareconfiguration of the information processing apparatus according to theembodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

The description is given in the following order.

1. Embodiment

1.1. With regard to overall configuration of medical observation system

1.2. With regard to configuration of medical observation apparatus

1.3. With regard to configuration of information processing apparatus

1.4. With regard to information processing method

1.5. With regard to hardware configuration

Embodiment

<With Regard to Overall Configuration of Medical Observation System>

First, the overall configuration of a medical observation system 1according to an embodiment of the present disclosure is brieflydescribed with reference to FIG. 1. FIG. 1 is an illustration diagramschematically showing the overall configuration of the medicalobservation system 1 according to the present embodiment.

As shown in FIG. 1, the medical observation system 1 according to thepresent embodiment includes a medical observation apparatus 10 forobserving the interior of a living body and an information processingapparatus 20.

The medical observation apparatus 10 is a medical apparatus thatobserves the interior of a living body S by a part of the medicalobservation apparatus 10 being inserted into the interior of the livingbody S or placed near an internal organ of the living body S. Examplesof the medical observation apparatus 10 like this include variousendoscope apparatuses such as rigid endoscopes, flexible endoscopes, andarthroscopes, and various microscope apparatuses.

The medical observation apparatus 10 according to the present embodimentapplies predetermined observation light to a part of the living body Sthat is an object to be observed and images the part of the living bodyS irradiated with observation light, and thereby generates a pluralityof biological images including a captured image of the living body inthe visible light region (a region falling under the wavelength range ofapproximately 400 nanometers to 800 nanometers).

The biological images may include, in addition to the captured image ofthe living body in the visible light region mentioned above, capturedimages of the living body in various wavelength ranges such as acaptured image of the living body in the near-infrared region (a regionfalling under the wavelength range of approximately 800 nanometers toapproximately 2.5 micrometers) and a captured image of the living bodyin the ultraviolet region, for example. Further, the biological imagesmay include other captured images of the living body such as an image inthe case where the living body is observed using ultrasonic waves and animage in the case where the living body is observed by optical coherencetomography (OCT), for example.

The plurality of biological images including a captured image of theliving body in the visible light region that are generated by themedical observation apparatus 10 are displayed at any time on a displaydevice such as a display connected to the medical observation apparatus10 via wire or wirelessly, and are outputted to the informationprocessing apparatus 20 according to the present embodiment described indetail later.

A specific configuration of the medical observation apparatus 10 isdescribed later.

The information processing apparatus 20 according to the presentembodiment analyzes captured images generated by the medical observationapparatus 10 and decides on an image relatively appropriate for workingas a recommended image, and recommends the recommended image to the userof the medical observation apparatus 10 (for example, a doctorperforming an operation or others). Thereby, in the informationprocessing apparatus 20 according to the present embodiment, thesituation at the time when the user observes the interior of the livingbody can be grasped, and an observation image in accordance with thesituation can be recommended as a recommended image to the user of themedical observation apparatus 10. As a result, the user of the medicalobservation apparatus 10 can be aware that there is an image that can bea relatively appropriate observation image in terms of the continuationof ongoing working, and the efficiency of working with the medicalobservation apparatus can be improved.

A specific configuration of the information processing apparatus 20 isdescribed later.

Although in FIG. 1 the case where the medical observation apparatus 10and the information processing apparatus 20 according to the presentembodiment exist as different apparatuses is described, the function ofthe information processing apparatus 20 described in detail later may beachieved as one function of the medical observation apparatus 10, as amatter of course.

<With Regard to Configuration of Medical Observation Apparatus 10>

Next, a specific configuration of the medical observation apparatus 10according to the present embodiment is briefly described with referenceto FIG. 1 to FIG. 2C. FIG. 2A to FIG. 2C are illustration diagramsschematically showing examples of the imaging unit included in themedical observation apparatus according to the present embodiment.

As mentioned above, the medical observation apparatus 10 according tothe present embodiment is a medical apparatus that observes the interiorof the living body S by a part of the medical observation apparatus 10being inserted into the interior of the living body S or placed near aninternal organ of the living body S.

As schematically shown in FIG. 1, the medical observation apparatus 10mainly includes a control unit 101, an observation unit 103, a lightsource unit 105, an imaging unit 107, and a display unit 109.

The control unit 101 is formed of, for example, a central processingunit (CPU), a read-only memory (ROM), a random access memory (RAM), etc.The control unit 101 is a unit that comprehensively controls the overallfunction of the medical observation apparatus 10, and collectivelycontrols the operating states of the observation unit 103, the lightsource unit 105, the imaging unit 107, and the display unit 109 includedin the medical observation apparatus 10. The control unit 101 is, forexample, a unit corresponding to a camera control unit (CCU) of amedical endoscope apparatus or a medical microscope apparatus. Further,the control unit 101 can output, at an arbitrary timing to theinformation processing apparatus 20 according to the present embodiment,a variety of information such as information concerning settingconditions that are set for each of the units constituting the medicalobservation apparatus 10 and information concerning the operating stateof each of the units constituting the medical observation apparatus 10.

The observation unit 103 is a unit of which a part is inserted into theinterior of the living body S or placed near an internal organ of theliving body S under the control by the control unit 101. The observationunit 103 causes an observation image of the interior of the living bodyS observed using observation light applied from the light source unit105 to form an image on the imaging unit 107 in a later stage. Theobservation unit 103 corresponds to, for example, an endoscope unit in amedical endoscope apparatus or a microscope unit in a medical microscopeapparatus. The observation unit 103 according to the present embodimentis not particularly limited, and known endoscope units, microscopeunits, and the like may be used.

The light source unit 105 is a unit that emits observation light forobserving the interior of the living body under the control by thecontrol unit 101, and is optically connected to the observation unit103. The light source unit 105 includes at least a visible light source(for example, a white light source or the like) for obtaining a capturedimage of the living body in the visible light region. The light sourceunit 105 may include not only such a visible light source but alsovarious known light sources such as a near-infrared light source forobtaining a captured image of the living body in the near-infraredregion, an ultraviolet light source, and various laser light sourcesthat emit specific types of visible light. The light source unit 105can, under the control by the control unit 101, switch the type ofobservation light emitted from these various light sources at anarbitrary timing. The light source unit 105 may include not only lightsources like the above but also a light source for achieving a specificfunction, such as a light source for OCT or a light source for distancemeasurement.

The imaging unit 107 is a unit that, under the control by the controlunit 101, captures an observation image of the interior of the livingbody S based on observation light from the light source unit 105 andgenerates image data of a plurality of biological images including acaptured image of the living body in the visible light region. Theimaging unit 107 is optically connected to the observation unit 103. Theimaging unit 107 according to the present embodiment performs imagingwhile distinguishing various types of observation light (for example,some types of light in various wavelength ranges such as visible light,near-infrared light, and ultraviolet light) from each other, and therebygenerates image data of a plurality of biological images.

In a common medical observation system, an image close to a situation ofdirect observation with the human eye is captured by using an imagingelement having sensitivity at wavelengths in the visible light regionand the image is appropriately developed, and the observation result isdisplayed on a display device as a normal observation image. Further, ina common medical observation system that can perform special lightobservation, in many cases various functions are achieved, such as, inaddition to a normal observation mode in which a captured image in thevisible light region is displayed, a fluorescence observation mode inwhich fluorescence occurring in the living body is observed using animaging element having sensitivity also at a wavelength in thenear-infrared region and a narrow wavelength image (narrow band imaging,NBI) observation mode in which a plurality of specific narrowwavelengths are combined to facilitate the distinction between bloodvessels at different depths from the surface of the skin.

However, special light observation like the above is enabled bycombining an illumination apparatus that selectively applies light of aspecific wavelength and an optical filter that transmits or reflectslight of a specific wavelength, and is therefore difficult to performsimultaneously with the normal observation mode of visible light.

Thus, in the medical observation apparatus 10 according to the presentembodiment, the imaging unit 107 like those shown in FIG. 2A to FIG. 2Cis provided, and thereby a captured image of the living body in thevisible light region and other captured images of the living body can beacquired simultaneously.

The imaging unit 107 may be, for example as shown in FIG. 2A, oneincluding any of various beam splitters such as a normal beam splitterand a polarizing beam splitter and two types of imaging elements of animaging element A and an imaging element B. In the case of the imagingunit 107 like that shown in FIG. 2A, an observation image from theobservation unit 103 is branched into two optical paths by the beamsplitter, and forms an image on each of the imaging element A and theimaging element B. Here, for example, an imaging element for visiblelight having sensitivity to light in the visible light region may beused as the imaging element A and an imaging element for near-infraredlight having sensitivity to light in the near-infrared region may beused as the imaging element B; thereby, a captured image of the livingbody in the visible light region and a captured image of the living bodyin the near-infrared region can be generated. Further, another knownimaging element may be used as the imaging element B, and therebyvarious captured images of the living body can be generated as well as acaptured image of the living body in the visible light region.

Although in FIG. 2A the case where one beam splitter is used is shown,two or more beam splitters may be used in combination, and thereby anobservation image can be branched into a larger number of optical pathsand a larger number of types of captured images of the living body canbe captured simultaneously.

Further, the imaging unit 107 may be, for example as shown in FIG. 2B,one including a spectral prism in which a plurality of (in FIG. 2B,three) optical prisms are joined to each other and a plurality of typesof imaging elements (in FIG. 2B, three types of an imaging element A toan imaging element C). In the case of the imaging unit 107 like thatshown in FIG. 2B, the joint surface between adjacent optical prisms maybe made to function as one of a beam splitter, a polarizing beamsplitter, and a wavelength selection filter, and thereby an observationimage from the observation unit 103 can be branched into a plurality ofoptical paths (in the case of FIG. 2B, three optical paths). Theobservation image from the observation unit 103 is branched into aplurality of optical paths by the spectral prism, and forms an image oneach imaging element provided at the end of each optical path. Here, forexample, an imaging element for visible light having sensitivity tolight in the visible light region may be used as the imaging element Aand an imaging element for near-infrared light having sensitivity tolight in the near-infrared region may be used as the imaging element B;thereby, a captured image of the living body in the visible light regionand a captured image of the living body in the near-infrared region canbe generated. Further, another known imaging element may be provided inthe position of the imaging element C as necessary, and thereby variouscaptured images of the living body can be generated as well as acaptured image of the living body in the visible light region and acaptured image of the living body in the near-infrared region.

In a spectral prism like that shown in FIG. 2B, the number of opticalprisms used may be increased, and thereby an observation image can bebranched into a still larger number of optical paths, and a largernumber of types of captured images of the living body can besimultaneously captured.

Further, the imaging unit 107 may be, as shown in FIG. 2C, one includingwhat is called a multispectral sensor. As schematically shown in FIG.2C, the multispectral sensor is a two-dimensional image sensor includingat least a pixel that detects the intensities of light in a plurality ofwavelength ranges at one time while distinguishing them for eachwavelength range. By using such a multispectral sensor, multispectraldata that are the intensities of light in a plurality of narrowwavelength ranges can be acquired in one pixel of the image.

It is preferable that all the pixels included in a multispectral sensorlike that schematically shown in FIG. 2C be a pixel that detects theintensities of light in a plurality of wavelength ranges at one timewhile distinguishing them for each wavelength range, but it is notnecessary that all the pixels be a pixel like the above that detects theintensities of light in a plurality of wavelength ranges at one timewhile distinguishing them for each wavelength range. This is because, byperforming various known interpolation processings using the output froma pixel like the above that detects the intensities of light in aplurality of wavelength ranges at one time while distinguishing them foreach wavelength range, the intensities of light in a plurality ofwavelength ranges can be calculated for each wavelength range even inthe position of a pixel that does not have a function like the above.

By using a multispectral sensor like that shown in FIG. 2C, the imagingunit 107 can be made smaller in size than the imaging unit 107 likethose shown in FIG. 2A and FIG. 2B, and the medical observationapparatus 10 can be further downsized.

The imaging unit 107 may be configured such that a beam splitter or aspectral prism like that shown in FIG. 2A or FIG. 2B and a multispectralsensor like that shown in FIG. 2C are combined with each other andthereby a larger number of types of biological images are generated, asa matter of course.

For various biological images generated by the imaging unit 107 like,for example, those shown in FIG. 2A to FIG. 2C, an appropriate image isselected by appropriate control performed by the control unit 101 inaccordance with the user manipulation by the user of the medicalobservation apparatus 10, and is displayed on a display screen providedin the display unit 109 in a later stage. The user of the medicalobservation apparatus 10 usually focuses attention on a captured imageof the living body in the visible light region as an observation imageof the interior of the living body; thus, in many cases mainly acaptured image of the living body in the visible light region isdisplayed on the display screen of the display unit 109 in a laterstage.

The imaging unit 107 outputs a plurality of biological images generatedin the above manner to the information processing apparatus 20 accordingto the present embodiment at any time under the control of the controlunit 101.

There may be a case where the number of imaging elements included in theimaging unit 107 according to the present embodiment is only one, suchas a case where the imaging unit 107 includes only one imaging elementhaving sensitivity in the visible light region to the near-infraredregion. Even in such a case, the light source unit 105 and the imagingunit 107 may, in cooperation with each other under the control of thecontrol unit 101, perform imaging while switching the wavelength rangeof light that forms an image on the imaging element; thereby, biologicalimages in a plurality of wavelength ranges can be generated.

The display unit 109 is a unit that displays at least one of theplurality of biological images generated by the imaging unit 107 to theuser of the medical observation apparatus 10 under the control by thecontrol unit 101. The number of display screens provided in the displayunit 109 is not particularly limited, and the display unit 109 mayinclude only one display screen or may include a plurality of displayscreens. The display unit 109 is not particularly limited, and a knowndisplay unit 109 may be used.

Further, information concerning a recommended image recommended by theinformation processing apparatus 20 described later is displayed on thedisplay screen of the display unit 109 according to the presentembodiment as appropriate.

In the above, an example of the configuration of the medical observationapparatus 10 according to the present embodiment is briefly describedwith reference to FIG. 1 to FIG. 2C. The medical observation apparatus10 according to the present embodiment may include, in addition tovarious units like those shown in FIG. 1, various other units such as anultrasonic wave oscillation apparatus and an ultrasonic wave detectionunit, for example.

<With Regard to Configuration of Information Processing Apparatus 20>

Next, the information processing apparatus 20 according to the presentembodiment is described in detail with reference to FIG. 3 to FIG. 9.

FIG. 3 is a block diagram schematically showing an example of theconfiguration of the information processing apparatus according to thepresent embodiment. FIG. 4 is a block diagram schematically showing anexample of the configuration of an image recommendation processing unitincluded in the information processing apparatus according to thepresent embodiment. FIG. 5 is an illustration diagram for describing asituation identifying database included in the information processingapparatus according to the present embodiment, and FIG. 6 is anillustration diagram showing an example of the absorption spectrum ofmain substances in a living tissue. FIG. 7 is an illustration diagramfor describing a recommended image database included in the informationprocessing apparatus according to the present embodiment. FIG. 8 andFIG. 9 are illustration diagrams schematically showing examples of thedisplay screen in the information processing apparatus according to thepresent embodiment.

With Regard to Overall Configuration of Information Processing Apparatus20

As mentioned above, the information processing apparatus 20 according tothe present embodiment is an apparatus that analyzes captured imagesgenerated by the medical observation apparatus 10 to decide on an imagerelatively appropriate for working as a recommended image and thatrecommends the recommended image to the user of the medical observationapparatus 10. As schematically shown in FIG. 3, the informationprocessing apparatus 20 mainly includes an information acquisition unit201, an image recommendation processing unit 203, an output processingunit 205, a display control unit 207, and a memory unit 209.

The information acquisition unit 201 is form of, for example, a CPU, aROM, a RAM, a communication device, etc. The information acquisitionunit 201 acquires a variety of information from the medical observationapparatus 10. For example, the information acquisition unit 201acquires, at any time, image data of a plurality of biological imagesincluding a captured image of the living body in the visible lightregion that are generated by the medical observation apparatus 10.Further, the information acquisition unit 201 can acquire a variety ofinformation concerning the medical observation apparatus 10 from themedical observation apparatus 10 (more specifically, the control unit101 etc. of the medical observation apparatus 10). Further, theinformation acquisition unit 201 can acquire, from the medicalobservation apparatus 10, information concerning the usage situation ofrecommended images, such as the result of the user's choice regardingimages recommended by the information processing apparatus 20 accordingto the present embodiment. Further, the information processing apparatus20 according to the present embodiment can also acquire informationconcerning various user manipulations made by the user of the medicalobservation apparatus 10, the manipulator of the information processingapparatus 20, or others.

Here, examples of the variety of information concerning the medicalobservation apparatus 10 include setting information concerning varioussettings of the medical observation apparatus 10, characteristicinformation concerning the characteristics of the medical observationapparatus 10, operating information concerning the operating state ofthe medical observation apparatus 10 and various peripheral devicescooperating with the medical observation apparatus 10, etc. Examples ofthe setting information concerning various settings of the medicalobservation apparatus 10 include the setting information of the lightsource unit 105 in the medical observation apparatus 10 (for example,information concerning the wavelength of observation light), theinformation of what type of peripheral device cooperates with themedical observation apparatus 10, etc. Examples of the characteristicinformation concerning the characteristics of the medical observationapparatus 10 include setting information concerning the sensor spectralcharacteristics of the imaging unit 107 in the medical observationapparatus 10, etc. Examples of the operating information concerning theoperating state of the medical observation apparatus 10 and variousperipheral devices cooperating with the medical observation apparatus 10include what type of light source is in operation as the light sourceunit 105, the operating situation of an energy device such as anelectric scalpel, the operating situation of a smoke extractionapparatus for discharging mist and smoke existing in the body cavity tothe outside of the body, etc.

On acquiring image data concerning a plurality of biological images fromthe medical observation apparatus 10, the information acquisition unit201 outputs the acquired image data to the image recommendationprocessing unit 203 described later. The information acquisition unit201 may associate time information concerning the date and time of theacquisition of image data etc. with the acquired image data, and mayrecord the image data as history information in the memory unit 209described later. Further, on the basis of information concerning theusage situation of recommended images acquired from the medicalobservation apparatus 10, the information acquisition unit 201 mayrecord the usage history of recommended images in the memory unit 209described later.

On acquiring a variety of information concerning the medical observationapparatus 10 like the above from the medical observation apparatus 10,the information acquisition unit 201 outputs the acquired variety ofinformation to the image recommendation processing unit 203 describedlater. The variety of information concerning the medical observationapparatus 10 may be notified from the control unit 101 of the medicalobservation apparatus 10 every time when image data of a plurality ofbiological images are outputted to the information processing apparatus20. Instead of every time when image data are outputted, the controlunit 101 of the medical observation apparatus 10 may notify a variety ofinformation concerning the medical observation apparatus 10 at thetiming when the content of any piece of information is changed.

The image recommendation processing unit 203 is formed of, for example,a CPU, a ROM, a RAM, etc. The image recommendation processing unit 203is a processing unit that uses image data and a variety of informationacquired by the information acquisition unit 201 to decide on arecommended image in accordance with the situation at the time pointwhen the interior of the living body is observed, and that performsimage recommendation processing that recommends the recommended image tothe user of the medical observation apparatus 10. The imagerecommendation processing unit 203 includes at least a recommendationunit that, on the basis of a plurality of biological images andsituation information concerning the situation at the time of observingthe interior of the living body or the surgical situation informationconcerning surgical characteristics at a time of observing an interiorof a living body, decides on at least one of the plurality of biologicalimages or at least one of images generated from the plurality ofbiological images as a recommended image in accordance with thesituation mentioned above.

A specific configuration of the image recommendation processing unit 203is described later.

On deciding on, using a plurality of biological images, a recommendedimage in accordance with the situation at the time point when theinterior of the living body is observed, the image recommendationprocessing unit 203 outputs information concerning the decided onrecommended image to the output processing unit 205 in a later stage.The image recommendation processing unit 203 may associate timeinformation concerning the date and time of obtainment of a processingresult regarding the decided on recommended image etc. with theprocessing result, and may store the processing result as historyinformation in the memory unit 209 in a later stage.

The output processing unit 205 is formed of, for example, a CPU, a ROM,a RAM, am output device, a communication device, etc. The outputprocessing unit 205 outputs a processing result regarding the decided onrecommended image outputted from the image recommendation processingunit 203 to the medical observation apparatus 10. The control unit 101of the medical observation apparatus 10 that has acquired the processingresult may cause the acquired processing result to be displayed on thedisplay unit 109, and can thereby recommend an observation image of theliving body in accordance with the situation to the user of the medicalobservation apparatus 10. The output processing unit 205 may output theprocessing result directly to the display unit 109 of the medicalobservation apparatus 10 in cooperation with the display control unit207 in a later stage.

The output processing unit 205 may output the processing resultregarding the recommended image outputted from the image recommendationprocessing unit 203 as a printed matter, or may output the processingresult as data to an external information processing apparatus, server,etc.

The display control unit 207 is formed of, for example, a CPU, a ROM, aRAM, an output device, a communication device, etc. The display controlunit 207 performs display control at the time of displaying variousresults outputted from the output processing unit 205 on an outputdevice such as a display included in the information processingapparatus 20, an output device provided outside an arithmetic processingdevice 200 (for example, the display unit 109 of the medical observationapparatus 10), or the like. Thereby, the user of the medical observationapparatus 10 can grasp various results on the spot.

The memory unit 209 is an example of the memory device included in theinformation processing apparatus 20, and is formed of a RAM, a storagedevice, etc. included in the information processing apparatus 20.Various databases used when the information processing apparatus 20according to the present embodiment performs image recommendationprocessing described in detail later are recorded in the memory unit209. Further, a variety of history information related to imagerecommendation processing described in detail later may be recorded inthe memory unit 209. Further, various parameters, situations of acertain processing still in progress, etc. that have to be saved whenthe information processing apparatus 20 according to the presentembodiment performs the processing, or various databases, programs, etc.are recorded in the memory unit 209 as appropriate. The informationacquisition unit 201, the image recommendation processing unit 203, theoutput processing unit 205, the display control unit 207, etc. canfreely perform read/write processing on the memory unit 209.

With Regard to Configuration of Image Recommendation Processing Unit 203

Next, the configuration of the image recommendation processing unit 203included in the information processing apparatus 20 according to thepresent embodiment is described in detail with reference to FIG. 4 toFIG. 7.

As mentioned above, the image recommendation processing unit 203according to the present embodiment uses image data and a variety ofinformation acquired by the information acquisition unit 201 to decideon a recommended image in accordance with the situation at the timepoint when the interior of the living body is observed, and performsimage recommendation processing that recommends the recommended image tothe user of the medical observation apparatus 10. As schematically shownin FIG. 4, the image recommendation processing unit 203 includes asituation identifying unit 211, a recommendation unit 213, and arecommended image generation unit 215.

The situation identifying unit 211 is formed of, for example, a CPU, aROM, a RAM, etc. The situation identifying unit 211 is a processing unitthat identifies the situation at the time when the interior of theliving body is observed on the basis of a plurality of biological imagesoutputted from the medical observation apparatus 10. More specifically,the situation identifying unit 211 identifies the situation at the timewhen the interior of the living body is observed on the basis of atleast one of the following pieces of information acquired by theinformation acquisition unit 201: setting information concerning thesetting of the medical observation apparatus 10, characteristicinformation concerning the characteristics of the medical observationapparatus 10, operating information concerning the operating state ofthe medical observation apparatus 10 and peripheral devices cooperatingwith the medical observation apparatus 10, a biological image, andsecondary information generated on the basis of a biological image.

The secondary information mentioned above generated on the basis of abiological image is not particularly limited, and any information may beused to the extent that it is information secondarily calculated byperforming various image processings or information processings on abiological image. Examples of the secondary information includeinformation concerning the spectral reflectance or the absorptioncoefficient of an organ and a biological substance existing in theinterior of the living body that can be calculated by focusing attentionon a captured image of the living body in a certain wavelength range,information concerning the oxygen saturation of blood, informationconcerning a recognition result obtained by performing image recognitionprocessing on a biological image using image information concerning anorgan existing in the interior of the living body and a medicalinstrument used for medical practice, etc.

More specifically, it is preferable that the situation identifying unit211 identify the situation at the time when the interior of the livingbody is observed with reference to a situation identifying database inwhich at least one of setting information concerning the setting of themedical observation apparatus 10, characteristic information concerningthe characteristics of the medical observation apparatus 10, operatinginformation concerning the operating state of the medical observationapparatus 10 and peripheral devices cooperating with the medicalobservation apparatus 10, a biological image, and secondary informationgenerated on the basis of a biological image, and the situation at thetime when the biological image is generated are associated with eachother in advance.

An example of the situation identifying database is schematically shownin FIG. 5.

As schematically shown in FIG. 5, situation-associated data in which avariety of information and a situation like the above are associated inadvance are stored in the situation identifying database stored in thememory unit 209 according to the present embodiment or the like. Thedata format of such situation-associated data is not particularlylimited; for example, a variety of information and a situation like theabove may be recorded in a data format such as a look-up table. Byreferring to situation-associated data while using an acquired varietyof information like the above, the situation identifying unit 211 canidentify the situation corresponding to the acquired variety ofinformation.

For example, in the case where reference to a variety of informationlike the above provides information that an energy device such as anelectric scalpel is in operation, information that a smoke extractionapparatus is in operation, or the like, the situation identifying unit211 may refer to situation-associated data while focusing attention oninformation like the above, and can thereby identify the fact that thevisibility of the visual field is reduced by smoke or mist generated dueto the energy device.

Further, information concerning the absorption coefficient or thespectral reflectance of an organ and a biological substance existing inthe interior of the living body may be stored in the situationidentifying database. For example, as shown in FIG. 6, each biologicalsubstance existing in the interior of the living body has an absorptioncoefficient peculiar to the biological substance. Therefore, abiological substance existing in the obtained biological images can bedetected by focusing attention on a captured image of the living body ina specific wavelength range included in the plurality of biologicalimages. Further, each organ and each biological substance existing inthe interior of the living body have its own peculiar spectralreflectance similarly to the absorption coefficient. Here, attention maybe given to a captured image of the living body in a specific wavelengthrange included in the plurality of biological images, the light sourcecharacteristics of the light source unit 105 of the medical observationapparatus 10, the sensor characteristics of the imaging unit 107, etc.,and thereby the spectral reflectance in the obtained biological imagescan be calculated. Thus, the situation identifying unit 211 can identifythe situation at the time when the interior of the living body isobserved by comparing the obtained information and the informationconcerning the spectral reflectance or the absorption coefficientrecorded in advance.

For example, in the case where the spectral reflectance or theabsorption coefficient corresponding to blood has been identified fromthe value of the spectral reflectance or the absorption coefficient byfocusing attention on information like the above (in particular, abiological image itself, secondary information obtained from abiological image, or the like), the situation identifying unit 211 canidentify the fact that there is a blood vessel and possibly bleeding hasoccurred. Further, in the case where data corresponding to water havebeen detected by focusing attention on information like the above (inparticular, a biological image itself, secondary information obtainedfrom a biological image, or the like), the situation identifying unit211 can infer that mist is generated in association with the use of anenergy device.

Further, image information concerning an organ existing in the interiorof the living body, a medical instrument used for medical practice, etc.may be stored in the situation identifying database. An organ, variousmedical instruments such as a blood flow obstruction medical instrumentor a blood vessel anastomosis medical instrument, etc. existing in thebiological images can be detected by performing image recognition on abiological image obtained from the medical observation apparatus 10using the pieces of image information mentioned above. Thereby, thesituation identifying unit 211 can infer the surgical site, the surgicalsituation, etc. The image recognition processing performed by thesituation identifying unit 211 is not particularly limited, and variousknown image recognition processings such as pattern recognition andrecognition by machine learning may be used as appropriate.

Other than various data like the above, various data for situationinference for inferring the situation may be stored in the situationidentifying database as appropriate.

On thus identifying the situation at the time when the interior of theliving body is observed while using the situation identifying databaseetc. as appropriate, the situation identifying unit 211 outputssituation information concerning the identified situation to therecommendation unit 213 in a later stage.

A situation identifying database like that shown in FIG. 5 may beupdated via various networks and external memory devices as appropriate.The situation identifying unit 211 may be configured such that thesituation identifying database is automatically updated by known machinelearning processing or the like.

The recommendation unit 213 is formed of, for example, a CPU, a ROM, aRAM, etc. On the basis of a plurality of biological images generated bythe medical observation apparatus 10 and situation information outputtedfrom the situation identifying unit 211 concerning the situation at thetime when the interior of the living body is observed, therecommendation unit 213 decides on at least one of the plurality ofbiological images or at least one of images generated from the pluralityof biological images as a recommended image in accordance with thesituation mentioned above. Further, the recommendation unit 213recommends the decided on recommended image to the user of the medicalobservation apparatus 10.

More specifically, it is preferable that the recommendation unit 213recommend a recommended image in accordance with the situation to theuser with reference to a recommended image database in which situationsat the time when the interior of the living body is observed and thetype of the recommended image that can be presented at the time of eachsituation are associated with each other in advance.

An example of the recommended image database is schematically shown inFIG. 7.

As schematically shown in FIG. 7, situations at the time when theinterior of the living body is observed and the type of the recommendedimage that can be presented at the time of each situation are recordedwhile being associated with each other in the recommended image databasestored in the memory unit 209 according to the present embodiment or thelike. In the case where the fact that a certain situation (situation A)has occurred is described in the situation information outputted fromthe situation identifying unit 211, the recommendation unit 213 mayrefer to the item related to situation A in the recommended imagedatabase, and can decide on the recommendation of a near-infrared imageas a recommended image, for example.

In the case where a plurality of recommended images are associated witha certain situation in the recommended image database, therecommendation unit 213 preferably recommends all the associatedrecommended images, rather than recommending the most suitable one fromamong the plurality of recommended images. In the case where a pluralityof situations that possibly have occurred are listed in the situationinformation outputted from the situation identifying unit 211, therecommendation unit 213 preferably recommends recommended imagescorresponding to all the situations described in the situationinformation, rather than recommending a recommended image related to oneof the plurality of situations. This is because it is preferable thatthe decision of what recommended image to use be made by the user of themedical observation apparatus 10 (that is, a doctor performing anoperation or others).

On deciding on a recommended image to be recommended to the user of themedical observation apparatus 10, the recommendation unit 213 outputsinformation concerning the decided on recommended image to the outputprocessing unit 205. The information concerning the recommended image isnot particularly limited, and may be, for example, at least one of textinformation showing the type of the recommended image and a thumbnailimage of the recommended image. The output processing unit 205 causesthe information concerning the recommended image to be outputted to theuser of the medical observation apparatus 10 or the manipulator of theinformation processing apparatus 20. Thereby, the user of the medicalobservation apparatus 10 or the manipulator of the informationprocessing apparatus 20 can decide whether to use the presentedrecommended image or not on the spot. Further, the output processingunit 205 may present the information concerning the recommended image tothe user of the medical observation apparatus 10 or the manipulator ofthe information processing apparatus 20 using a means other thancharacters or images, such as sound or vibration.

On deciding on the type of the recommended image to be recommended tothe user of the medical observation apparatus 10, the recommendationunit 213 requests the recommended image generation unit 215 in a laterstage to generate the decided on recommended image, as necessary.

In the case where the user of the medical observation apparatus 10 haschosen reference to a recommended image, the recommendation unit 213causes the chosen recommended image to be appropriately presented to theuser of the medical observation apparatus 10 via the output processingunit 205. The recommendation unit 213 may assist the choice by the userof the medical observation apparatus 10 or others by recording thehistory of the result of choice by the user of the medical observationapparatus 10 or others in the memory unit 209 or the like as appropriateand causing also the history information to be displayed at the time ofthe next recommendation.

A recommended image database like that shown in FIG. 7 may be updatedvia various networks or external memory devices as appropriate. Therecommendation unit 213 may be configured such that the recommendedimage database is automatically updated by known machine learningprocessing or the like.

The recommended image generation unit 215 is formed of, for example, aCPU, a ROM, a RAM, etc. The recommended image generation unit 215 usesat least one of at least part of the plurality of biological images andan image in which secondary information generated by analyzing at leastone of the plurality of biological images is visualized, and generates arecommended image to be recommended by the recommendation unit 213.

For example, in the case where an observation image in a specificwavelength range (for example, the near-infrared region) is recommendedas a recommended image by the recommendation unit 213, the recommendedimage generation unit 215 may use a captured image of the living body inthe corresponding wavelength range out of the plurality of biologicalimages, and can thereby generate an observation image, a thumbnail imagethereof, or the like in the relevant wavelength range as a recommendedimage. Further, in the case where oxygen saturation-visualizedinformation in which the distribution of oxygen saturation identifiableas secondary information of a biological image is visualized, such asthat shown by the database of FIG. 7, is a recommended image, thesecondary information may be used to generate a relevant recommendedimage, a thumbnail image thereof, or the like. Other than these images,a known method such as the principle of NBI may be used, and therebyvarious known images such as a blood vessel-highlighted image in whichthe existence of a blood vessel is highlighted and an image showing themanner of blood flowing, a thumbnail image thereof, or the like can begenerated as appropriate.

Further, attention is given to the case where a captured image of theliving body in the visible light region and a captured image of theliving body in the near-infrared region (for example, a fluorescenceimage in the near-infrared region or the like) are included as aplurality of biological images. Here, in the fluorescence image in thenear-infrared region, in many cases the portion corresponding tofluorescence is displayed in a predetermined color tone and the portionnot corresponding to fluorescence (the portion in which the luminancevalue is zero or close to zero) is displayed in black. Even when such afluorescence image is presented as a recommended image, it is highlylikely that the positional relationship between internal organs etc. ishard to understand. Thus, the recommended image generation unit 215first changes the color tone of the area in the captured image of theliving body in the near-infrared region where light in the near-infraredregion has been detected (for example, the portion corresponding tofluorescence) to a predetermined color tone (for example, green color,which is a color tone not existing in the living body), and changes theluminance value of the area where light in the near-infrared region hasnot been detected to zero. After that, the recommended image generationunit 215 may generate a superimposed image in which the captured imageof the living body in the near-infrared region after change issuperimposed on the captured image of the living body in the visiblelight region, and may use the superimposed image as a recommended image.

In the above, an example of the function of the information processingapparatus 20 according to the present embodiment is described. Eachstructural element mentioned above may be configured using ageneral-purpose member or circuit, or may be configured using a hardwaremember dedicated to the function of each structural element. It is alsopossible for all the functions of the structural elements to beperformed by a CPU or the like. Thus, the configuration for use may bealtered in accordance with the technical level on every occasion ofimplementing the present embodiment, as appropriate.

A computer program for performing each function of the informationprocessing apparatus according to the present embodiment like the abovemay be created and mounted on a personal computer or the like. Further,a computer-readable recording medium in which such a computer program isstored can be provided. The recording medium is, for example, a magneticdisk, an optical disk, a magneto-optical disk, a flash memory, or thelike. The computer program mentioned above may be distributed via anetwork without using a recording medium, for example.

With Regard to Examples of Presentation of Recommended Image

Next, methods for presenting a recommended image decided on by therecommendation unit 213 are briefly described with reference to FIG. 8and FIG. 9.

FIG. 8 schematically shows examples of the display screen viewed by theuser of the medical observation apparatus 10 or the manipulator of theinformation processing apparatus 20, such as the display unit 109 of themedical observation apparatus 10 or a display device of the informationprocessing apparatus 20 (not shown).

In general, when various medical practices such as an operation areperformed, it is probable that, as schematically shown in the diagram atthe upper left of FIG. 8, a captured image of the living body in thevisible light region (a visible light image) is displayed as normaldisplay on the display screen by the control unit 101 of the medicalobservation apparatus 10 or the display control unit 207 of theinformation processing apparatus 20. Here, when a recommended image inaccordance with the situation is proposed by the recommendation unit213, for example as schematically shown in the diagram at the upperright of FIG. 8, the recommendation unit 213 may cause text informationshowing the type of the recommended image to be displayed so as to besuperimposed on the visible light image via the output processing unit205. Alternatively, for example as schematically shown in the diagram atthe lower left of FIG. 8, the recommendation unit 213 may cause, insteadof only text information, an text image showing the type of therecommended image and a thumbnail image of the recommended image to bedisplayed together, or may cause a thumbnail image of the recommendedimage to be singly displayed in the display format of what is calledpicture-in-picture. Alternatively, as schematically show in the diagramat the lower right of FIG. 8, the recommendation unit 213 may cause adisplay region for displaying information concerning recommended imagesto be provided around the visible light image, and may cause textinformation showing the types of the recommended images, thumbnailimages of the recommended images, text information describing therecommended images, etc. to be displayed together in the display region.

By generating a display screen like those shown in FIG. 8, the user ofthe medical observation apparatus 10 is enabled to choose whether tocause a recommended image to be displayed on the display screen or not.

Further, instead of using one display screen like those shown in FIG. 8,as schematically shown in FIG. 9 a recommended image like the above andinformation related to the recommended image may be caused to bedisplayed on, not a main monitor on which a visible light image isdisplayed, but a sub-monitor provided separately from the main monitor.

The display of information concerning recommended images like thoseshown in FIG. 8 and FIG. 9 may be caused to disappear every time whenthe user of the medical observation apparatus 10 does not choose the useof the recommended image. Further, the display of information concerninga recommended image like those shown in FIG. 8 and FIG. 9 may be causedto automatically disappear after a certain time has elapsed. Further,the display of information concerning a recommended image like thoseshown in FIG. 8 and FIG. 9 may be recorded by various methods so that itcan be checked by reproduction during the operation or after theoperation.

On the other hand, in the case where, based on the display ofinformation concerning a recommended image like those shown in FIG. 8and FIG. 9, the user of the medical observation apparatus 10 has chosenthe use of the recommended image, the chosen recommended image isdisplayed on the display screen. The timing of the re-switching from therecommended image to the normal visible light image is not particularlylimited, and the re-switching is performed in accordance with the usermanipulation etc. by the user of the medical observation apparatus 10,as appropriate.

In the above, methods for presenting a recommended image decided on bythe recommendation unit 213 are briefly described.

With Regard to Specific Examples of Recommended Image

In the following, the recommended image is briefly described with somespecific examples.

Case 1

In the case where, for example, bleeding has occurred during anoperation with the medical observation apparatus 10, it is importantthat the user of the medical observation apparatus 10 quickly find thebleeding point and perform hemostasic treatment. However, when there isa large amount of bleeding, there is a case where the surroundings arebloodstained and it is difficult to look for the bleeding point.

In this event, it is assumed that blood has been recognized and thesituation of ongoing bleeding has been identified by situationidentifying processing by the situation identifying unit 211 (forexample, the processing of analyzing a captured image in a specificwavelength range). On the other hand, the absorption coefficient ofblood is low in the near-infrared region as shown in FIG. 6, and lightin the near-infrared region is transmitted through blood. Thus, therecommendation unit 213 refers to the recommended image database etc.,and selects, as a recommended image, an observation image in awavelength range in which the absorption coefficient of blood isrelatively low. By using such a recommended image, it becomes easy forthe user of the medical observation apparatus 10 to visually identifythe surface of the living body that is buried in blood and hard to seeand to find the bleeding point.

Case 2

When, for example, an energy device such as an electric scalpel is used,there is a case where the interior of the body cavity is covered withsurgical smoke or mist and the visibility is worsened.

In this event, it is assumed that the situation of surgical smoke ormist being generated has been identified by situation identifyingprocessing by the situation identifying unit 211 (for example, theprocessing of analyzing a captured image in a specific wavelength range,or information concerning the operating situation of various devices).In this case, the recommendation unit 213 refers to the recommendedimage database etc., and selects, as a recommended image, an observationimage in the near-infrared region in which the absorption coefficient ofsurgical smoke and mist is relatively low. By using such a recommendedimage, the user of the medical observation apparatus 10 can continue themedical practice without suspending the working due to surgical smoke ormist.

Further, in the case where it has been assessed that, after the lapse ofa certain period of time of a situation like the above, the smokeextraction device etc. operate normally and thereby the surgical smoke,mist, and the like have disappeared, the recommendation unit 213 mayswitch the display from the observation image in the near-infraredregion to the normal observation image in the visible light region.Thus, the user of the medical observation apparatus 10 can continue themedical practice using a familiar image.

Case 3

When, for example, cutting off a blood vessel safely, the doctorcompletely obstructs the blood by attaching a blood flow obstructionmedical instrument such as a clip to the blood vessel, and then cuts offthe blood vessel with a predetermined surgical instrument. In thisevent, there is a case where the attachment of the blood flowobstruction medical instrument such as a clip is insufficient and theblood flow is not completely obstructed. If the blood vessel is cut offin the situation where the blood flow is not completely obstructed,unintentional bleeding occurs. Similarly, it is necessary toappropriately anastomose the blood vessels or tissues using a bloodvessel anastomosis medical instrument such as a stapler and to checkwhether the blood flow is resumed or not.

In this event, in the case where situation information that the bloodflow obstruction medical instrument or the blood vessel anastomosismedical instrument is detected has been generated by situationidentifying processing by the situation identifying unit 211 (forexample, the image recognition processing of the clip, the stapler, orthe like), the recommendation unit 213 may calculate the oxygensaturation as secondary information based on a biological image by aknown method, and may use, as a recommended image, an image in which thedistribution of oxygen saturation is visualized.

Instead of an image in which the distribution of oxygen saturation isvisualized, a captured image of the living body in the near-infraredregion corresponding to the distribution of blood may be used as arecommended image. For example, a fluorescent reagent called indocyaninegreen (ICG) has the property of binding to plasma protein in blood andthereby emitting fluorescence of 830 nanometers by means of excitationlight of 760 nanometers. Thus, in the case where a situation like theabove is grasped, the recommendation unit 213 may first recommend theuse of the fluorescent reagent to the doctor who is the user of themedical observation apparatus 10, and may recommend a near-infraredimage at a wavelength of around 830 nanometers generated due to thefluorescent reagent, as an observation image in the near-infrared regioncorresponding to the distribution of blood.

Case 4

There may be a case where, for example, during an operation with themedical observation apparatus 10 the user of the medical observationapparatus 10 wants to know information concerning the running of a bloodvessel or the depth of a blood vessel; but a common captured image inthe visible light region may not allow such information to besufficiently obtained.

In this event, when the fact that a blood vessel exists in a biologicalimage is identified by situation identifying processing by the situationidentifying unit 211 (for example, the processing of analyzing acaptured image in a specific wavelength range), the recommendation unit213 may use a similar means to the principle of NBI, and may recommendan image in a wavelength range suitable for the visual identification ofthe blood vessel as a blood vessel-highlighted image.

In the above, the recommended image is briefly described with somespecific examples; but the recommended image in the informationprocessing apparatus 20 according to the present embodiment is notlimited to the examples described above, and various known images may beused as the recommended image.

<With Regard to Information Processing Method>

Next, a flow of an information processing method according to thepresent embodiment is briefly described with reference to FIG. 10. FIG.10 is a flow chart showing an example of the flow of the informationprocessing method according to the present embodiment.

In the information processing method according to the presentembodiment, first, biological images are generated by the medicalobservation apparatus 10, and are outputted to the informationprocessing apparatus 20. On acquiring the data of the biological imagesoutputted from the medical observation apparatus 10 (step S101), theinformation acquisition unit 201 of the information processing apparatus20 outputs the acquired data to the image recommendation processing unit203.

The situation identifying unit 211 of the image recommendationprocessing unit 203 identifies the situation at the time when theinterior of the living body is observed by a method like that describedabove (step S103), and outputs situation information concerning theidentified situation to the recommendation unit 213.

The recommendation unit 213 uses the situation information outputtedfrom the situation identifying unit 211 etc., and assesses whether thereis an image recommendable to the user of the medical observationapparatus 10 or others or not (step S105). In the case where there is norecommendable image (step S105—NO), step S115 described later isperformed. On the other hand, in the case where there is a recommendableimage (step S105—YES), the recommendation unit 213 causes informationconcerning the recommended image to be notified to the user of themedical observation apparatus 10 by a method like that described above(step S107).

Here, in the case where the user of the medical observation apparatus 10or others have not chosen the recommended image (step S109—NO), stepS115 described later is performed. On the other hand, in the case wherethe user of the medical observation apparatus 10 or others have chosenthe recommended image (step S109—YES), the recommended image isgenerated by the recommended image generation unit 215 as necessary(step S111), and the output processing unit 205, the control unit 101 ofthe medical observation apparatus 10, etc. work in cooperation; thereby,the recommended image is displayed on a screen (step S113). After that,the procedure returns to step S101, and the information processingmethod according to the present embodiment is continued.

On the other hand, in the case where in step S105 it has been assessedthat there is no recommendable image and in the case where in step S109the recommended image has not been chosen, a captured image of theliving body in the visible light region is generated (step S115), andthe output processing unit 205, the control unit 101 of the medicalobservation apparatus 10, etc. work in cooperation; thereby, the visiblelight image is displayed on a screen (step S117). After that, theprocedure returns to step S101, and the information processing methodaccording to the present embodiment is continued.

In the above, a flow of the information processing method according tothe present embodiment is briefly described with reference to FIG. 10.

<Hardware Configuration>

Next, the hardware configuration of the information processing apparatus20 according to the embodiment of the present disclosure will bedescribed in detail with reference to FIG. 11. FIG. 11 is a blockdiagram for illustrating the hardware configuration of the informationprocessing apparatus 20 according to the embodiment of the presentdisclosure.

The information processing apparatus 20 mainly includes a CPU 901, a ROM903, and a RAM 905. Furthermore, the information processing apparatus 20also includes a host bus 907, a bridge 909, an external bus 911, aninterface 913, an input device 915, an output device 917, a storagedevice 919, a drive 921, a connection port 923, and a communicationdevice 925.

The CPU 901 serves as an arithmetic processing apparatus and a controldevice, and controls the overall operation or a part of the operation ofthe information processing apparatus 20 according to various programsrecorded in the ROM 903, the RAM 905, the storage device 919, or aremovable recording medium 927. The ROM 903 stores programs, operationparameters, and the like used by the CPU 901. The RAM 905 primarilystores programs that the CPU 901 uses and parameters and the likevarying as appropriate during the execution of the programs. These areconnected with each other via the host bus 907 configured from aninternal bus such as a CPU bus or the like.

The host bus 907 is connected to the external bus 911 such as a PCI(Peripheral Component Interconnect/Interface) bus via the bridge 909.

The input device 915 is an operation means operated by a user, such as amouse, a keyboard, a touch panel, buttons, a switch and a lever. Also,the input device 915 may be a remote control means (a so-called remotecontrol) using, for example, infrared light or other radio waves, or maybe an externally connected apparatus 929 such as a mobile phone or a PDAconforming to the operation of the information processing apparatus 20.Furthermore, the input device 915 generates an input signal based on,for example, information which is input by a user with the aboveoperation means, and is configured from an input control circuit foroutputting the input signal to the CPU 901. The user can input variousdata to the information processing apparatus 20 and can instruct theinformation processing apparatus 20 to perform processing by operatingthis input apparatus 915.

The output device 917 is configured from a device capable of visually oraudibly notifying acquired information to a user. Examples of suchdevice include display devices such as a CRT display device, a liquidcrystal display device, a plasma display device, an EL display deviceand lamps, audio output devices such as a speaker and a headphone, aprinter, a mobile phone, a facsimile machine, and the like. For example,the output device 917 outputs a result obtained by various processingsperformed by the information processing apparatus 20. More specifically,the display device displays, in the form of texts or images, a resultobtained by various processes performed by the information processingapparatus 20. On the other hand, the audio output device converts anaudio signal such as reproduced audio data and sound data into an analogsignal, and outputs the analog signal.

The storage device 919 is a device for storing data configured as anexample of a storage unit of the information processing apparatus 20 andis used to store data. The storage device 919 is configured from, forexample, a magnetic storage device such as a HDD (Hard Disk Drive), asemiconductor storage device, an optical storage device, or amagneto-optical storage device. This storage device 919 stores programsto be executed by the CPU 901, various data, and various data obtainedfrom the outside.

The drive 921 is a reader/writer for recording medium, and is embeddedin the information processing apparatus 20 or attached externallythereto. The drive 921 reads information recorded in the attachedremovable recording medium 927 such as a magnetic disk, an optical disk,a magneto-optical disk, or a semiconductor memory, and outputs the readinformation to the RAM 905. Furthermore, the drive 921 can write in theattached removable recording medium 927 such as a magnetic disk, anoptical disk, a magneto-optical disk, or a semiconductor memory. Theremovable recording medium 927 is, for example, a DVD medium, an HD-DVDmedium, or a Blu-ray (registered trademark) medium. The removablerecording medium 927 may be a CompactFlash (CF; registered trademark), aflash memory, an SD memory card (Secure Digital Memory Card), or thelike. Alternatively, the removable recording medium 927 may be, forexample, an IC card (Integrated Circuit Card) equipped with anon-contact IC chip or an electronic appliance.

The connection port 923 is a port for allowing devices to directlyconnect to the information processing apparatus 20. Examples of theconnection port 923 include a USB (Universal Serial Bus) port, anIEEE1394 port, a SCSI (Small Computer System Interface) port, and thelike. Other examples of the connection port 923 include an RS-232C port,an optical audio terminal, an HDMI (High-Definition MultimediaInterface) port, and the like. By the externally connected apparatus 929connecting to this connection port 923, the information processingapparatus 20 directly obtains various data from the externally connectedapparatus 929 and provides various data to the externally connectedapparatus 929.

The communication device 925 is a communication interface configuredfrom, for example, a communication device for connecting to acommunication network 931. The communication device 925 is, for example,a wired or wireless LAN (Local Area Network), Bluetooth (registeredtrademark), a communication card for WUSB (Wireless USB), or the like.Alternatively, the communication device 925 may be a router for opticalcommunication, a router for ADSL (Asymmetric Digital Subscriber Line), amodem for various communications, or the like. This communication device925 can transmit and receive signals and the like in accordance with apredetermined protocol such as TCP/IP on the Internet and with othercommunication devices, for example. The communication network 931connected to the communication device 925 is configured from a networkand the like, which is connected via wire or wirelessly, and may be, forexample, the Internet, a home LAN, infrared communication, radio wavecommunication, satellite communication, or the like.

Heretofore, an example of the hardware configuration capable ofrealizing the functions of the information processing apparatus 20according to the embodiment of the present disclosure has been shown.Each of the structural elements described above may be configured usinga general-purpose material, or may be configured from hardware dedicatedto the function of each structural element. Accordingly, the hardwareconfiguration to be used can be changed as appropriate according to thetechnical level at the time of carrying out the present embodiment.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art based on the description of this specification.

Additionally, the present technology may also be configured as below.

-   (1)-   A medical information processing apparatus comprising:-   processing circuitry configured to, based on surgical situation    information concerning surgical characteristics at a time of    observing an interior of a living body, select at least one of a    plurality of biological images each having a different wavelength    range or at least one of secondary images generated from the    plurality of biological images each having a different wavelength    range, as a recommended image.-   (2)-   The medical information processing apparatus according to (1),-   wherein the plurality of biological images include at least a    captured image of the living body in a visible light region, and-   wherein the processing circuitry is further configured to-   perform display control of a display screen on which the plurality    of biological images are displayed, and-   switch the display screen to display the captured image of the    living body in the visible light region and the recommended image on    the display screen in response to a manual input.-   (3)-   The medical information processing apparatus according to (1),-   wherein the processing circuitry is further configured to access a    recommended image database which associates different surgical    characteristics at the time of observing the interior of the living    body with different recommended images, and selects the recommended    image based on respective surgical characteristics.-   (4)-   The medical information processing apparatus according to (1),    wherein the processing circuitry is further configured to generate    the recommended image using at least one of:-   at least some of the plurality of biological images and secondary    information generated by analyzing at least one of the plurality of    biological images.-   (5)-   The medical information processing apparatus according to (4),-   wherein the plurality of biological images include a first captured    image of the living body in a visible light region and a second    captured image of the living body in a near-infrared region, and-   the processing circuitry is further configured to change a color    tone of an area of the second captured image in which light, in the    near-infrared region, has been detected to a predetermined color    tone in the display.-   (6)-   The medical information processing apparatus according to (1),-   wherein the plurality of biological images include at least a    captured image of the living body in a visible light region, and-   the processing circuitry instructs display of information regarding    the recommended image on a display screen on which the captured    image of the living body in the visible light region is also    displayed.-   (7)-   The medical information processing apparatus according to (6),    wherein the information concerning the recommended image is at least    one of text information showing a type of the recommended image and    a thumbnail image of the recommended image.-   (8)-   The medical information processing apparatus according to (1),    wherein the processing circuitry is configured to instruct display    of the recommended image on a first display screen different from a    second display screen on which a captured image of the living body    in a visible light region out of the plurality of biological images,    is displayed.-   (9)-   The medical information processing apparatus according to (1),    wherein the processing circuitry is further configured to identify    the surgical characteristics at the time that the interior of the    living body is observed based on at least one of:-   settings of a medical observation apparatus used to observe the    interior of the living body, characteristics of the medical    observation apparatus, an operating state of the medical observation    apparatus or a peripheral device cooperating with the medical    observation apparatus, the plurality of biological images, and    secondary information generated based on the plurality of biological    images.-   (10)-   The medical information processing apparatus according to (9),-   wherein the processing circuitry is further configured to identify    the surgical characteristics at the time that the interior of the    living body is observed with reference to a surgical characteristics    identification database in which at least one of: settings of a    medical observation apparatus used to observe the interior of the    living body, characteristics of the medical observation apparatus,    an operating state of the medical observation apparatus or a    peripheral device cooperating with the medical observation    apparatus, the plurality of biological images, and secondary    information generated based on the plurality of biological images,    is associated with surgical characteristics in effect at a time that    the plurality of biological images are generated.-   (11)-   The medical information processing apparatus according to (10),-   wherein information concerning a spectral reflectance or an    absorption coefficient of an organ and a biological substance    existing in the interior of the living body is further recorded in    the surgical characteristics identification database, and-   wherein the processing circuitry is further configured to identify    the surgical characteristics at the time that the interior of the    living body is being observed based on the information concerning    the spectral reflectance or the absorption coefficient of the organ    and the biological substance existing in the interior of the living    body.-   (12)-   The medical information processing apparatus according to (10),-   wherein first image information concerning an organ in the interior    of the living body or a medical instrument is further recorded in    the surgical characteristics identification database, and-   the processing circuitry is further configured to identify the    surgical characteristics at the time when the interior of the living    body is observed, based on the first image information.-   (13)-   The medical information processing apparatus according to (1),-   wherein a first captured image of the living body in a visible light    region and a second captured image of the living body in a    near-infrared region, are included in the plurality of biological    images, and-   the processing circuitry is configured to select the first captured    image of the living body in the near-infrared region as the    recommended image in a case that the surgical characteristics    include that blood is detected.-   (14)-   The medical information processing apparatus according to (1),-   wherein at least a first captured image of the living body in a    visible light region and a second captured image of the living body    in a near-infrared region are included in the plurality of    biological images, and-   wherein the processing circuitry is configured to select the second    captured image of the living body in the near-infrared region as the    recommended image in a case that the surgical characteristics    include that smoke or mist is detected or a medical energy device is    in use.-   (15)-   The medical information processing apparatus according to (14),-   wherein the processing circuitry is configured to instruct a display    screen to be switched from the recommended image to the first    captured image of the living body in the visible light region in a    case that the surgical characteristics include that no smoke or mist    is detected and that use of the medical energy device is finished.-   (16)-   The medical information processing apparatus according to (1),-   wherein the processing circuitry is configured to select, as the    recommended image, an image of the secondary images in which a    distribution of oxygen saturation generated as secondary    information, based on the plurality of biological images, is    visualized, or a captured image of the living body in a    near-infrared region of the plurality of biological images, that    corresponds to a distribution of blood, in a case that the surgical    characteristics include that a blood flow obstruction medical    instrument or a blood vessel anastomosis medical instrument, is    detected.-   (17)-   The medical information processing apparatus according to (1),-   wherein on the processing circuitry is configured to select a    captured image of the living body of the plurality of biological    images having a wavelength range permitting visual identification of    a blood vessel, as the recommended image in a case that the surgical    characteristics include that an existence of a blood vessel is    detected.-   (18)-   The medical information processing apparatus according to (1),    further comprising:-   a medical observation apparatus that includes a multispectral sensor    including at least a pixel configured to detect intensities of light    in a plurality of wavelength ranges at one time while distinguishing    intensities for each wavelength range,-   wherein a biological image for each of the plurality of wavelength    ranges is generated from an output result of the multispectral    sensor as the plurality of biological images.-   (19)-   The medical information processing apparatus according to (1),    further comprising:-   a medical observation apparatus that includes a plurality of imaging    elements corresponding to specific wavelength ranges,-   wherein a biological image for each of a plurality of wavelength    ranges is generated from an output result of the imaging elements as    the plurality of biological images.-   (20)-   The medical information processing apparatus according to (1),    further comprising:-   a medical observation apparatus that generates a biological image    for each of a plurality of wavelength ranges as the plurality of    biological images, by performing imaging while switching a    wavelength range of light that forms an image on an imaging element    included in the medical observation apparatus.-   (21)-   The information processing apparatus according to (1), further    comprising:-   a medical observation apparatus that is a medical endoscope or a    medical microscope.-   (22)-   A medical information processing method including:-   selecting, based on surgical situation information concerning    surgical characteristics at a time of observing an interior of a    living body, at least one of a plurality of biological images each    having a different wavelength range or at least one of secondary    images generated from the plurality of biological images each having    a different wavelength range, as a recommended image.-   (23)-   A non-transitory computer readable medium having stored therein a    program that when executed by a computer causes the computer to    execute a medical information processing method, comprising:-   selecting, based on surgical situation information concerning    surgical characteristics at a time of observing an interior of a    living body, at least one of a plurality of biological images each    having a different wavelength range or at least one of secondary    images generated from the plurality of biological images each having    a different wavelength range, as a recommended image.-   (24)-   A medical observation system including:-   a medical observation apparatus including first processing circuitry    configured to generate a plurality of biological images by capturing    images of an interior and exterior of a living body in a plurality    of wavelength ranges; and-   an information processing apparatus including-   second processing circuitry configured to, based on surgical    situation information concerning surgical characteristics at a time    of observing the interior of the living body, select at least one of    the plurality of biological images having different wavelength    ranges or at least one of secondary images generated from the    plurality of biological images, as a recommended image.

REFERENCE SIGNS LIST

-   -   1 medical observation system    -   10 medical observation apparatus    -   20 information processing apparatus    -   101 control unit (processing circuitry)    -   103 observation unit    -   105 light source unit (light source)    -   107 imaging unit    -   109 display unit (display)    -   201 information acquisition unit (processing circuitry)    -   203 image recommendation processing unit (processing circuitry)    -   205 output processing unit (processing circuitry)    -   207 display control unit (processing circuitry)    -   209 memory unit (memory)    -   211 situation identifying unit (processing circuitry)    -   213 recommendation unit (processing circuitry)    -   215 recommended image generation unit (processing circuitry)

The invention claimed is:
 1. A medical information processing apparatus comprising: processing circuitry configured to based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, select at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein the plurality of biological images include a first captured image of the living body in a visible light region and a second captured image of the living body in a near-infrared region; and change a color tone of an area of the second captured image in which light, in the near-infrared region, has been detected to a predetermined color tone in the display.
 2. The medical information processing apparatus according to claim 1, wherein the processing circuitry is further configured to perform display control of a display screen on which the plurality of biological images are displayed, and switch the display screen to display the first captured image of the living body in the visible light region and the recommended image on the display screen in response to a manual input.
 3. The medical information processing apparatus according to claim 1, wherein the processing circuitry is further configured to access a recommended image database which associates different surgical characteristics at the time of observing the interior of the living body with different recommended images, and selects the recommended image based on respective surgical characteristics.
 4. The medical information processing apparatus according to claim 1, wherein the processing circuitry is further configured to generate the recommended image using at least one of: at least some of the plurality of biological images and secondary information generated by analyzing at least one of the plurality of biological images.
 5. The medical information processing apparatus according to claim 1, wherein the plurality of biological images include at least a captured image of the living body in a visible light region, and the processing circuitry instructs display of information regarding the recommended image on a display screen on which the captured image of the living body in the visible light region is also displayed.
 6. The medical information processing apparatus according to claim 5, wherein the information concerning the recommended image is at least one of text information showing a type of the recommended image and a thumbnail image of the recommended image.
 7. The medical information processing apparatus according to claim 1, wherein the processing circuitry is configured to instruct display of the recommended image on a first display screen different from a second display screen on which the first captured image of the living body in a visible light region is displayed.
 8. The medical information processing apparatus according to claim 1, wherein the processing circuitry is further configured to identify the surgical characteristics at the time that the interior of the living body is observed based on at least one of: settings of a medical observation apparatus used to observe the interior of the living body, characteristics of the medical observation apparatus, an operating state of the medical observation apparatus or a peripheral device cooperating with the medical observation apparatus, the plurality of biological images, and secondary information generated based on the plurality of biological images.
 9. The medical information processing apparatus according to claim 8, wherein the processing circuitry is further configured to identify the surgical characteristics at the time that the interior of the living body is observed with reference to a surgical characteristics identification database in which at least one of: settings of a medical observation apparatus used to observe the interior of the living body, characteristics of the medical observation apparatus, an operating state of the medical observation apparatus or a peripheral device cooperating with the medical observation apparatus, the plurality of biological images, and secondary information generated based on the plurality of biological images, is associated with surgical characteristics in effect at a time that the plurality of biological images are generated.
 10. The medical information processing apparatus according to claim 9, wherein information concerning a spectral reflectance or an absorption coefficient of an organ and a biological substance existing in the interior of the living body is further recorded in the surgical characteristics identification database, and wherein the processing circuitry is further configured to identify the surgical characteristics at the time that the interior of the living body is being observed based on the information concerning the spectral reflectance or the absorption coefficient of the organ and the biological substance existing in the interior of the living body.
 11. The medical information processing apparatus according to claim 9, wherein first image information concerning an organ in the interior of the living body or a medical instrument is further recorded in the surgical characteristics identification database, and the processing circuitry is further configured to identify the surgical characteristics at the time when the interior of the living body is observed, based on the first image information.
 12. The medical information processing apparatus according to claim 1, wherein the processing circuitry is configured to select the first captured image of the living body in the near-infrared region as the recommended image in a case that the surgical characteristics include that blood is detected.
 13. The medical information processing apparatus according to claim 1, further comprising: a medical observation apparatus that includes a plurality of imaging elements corresponding to specific wavelength ranges, wherein a biological image for each of a plurality of wavelength ranges is generated from an output result of the imaging elements as the plurality of biological images.
 14. The medical information processing apparatus according to claim 1, further comprising: a medical observation apparatus that generates a biological image for each of a plurality of wavelength ranges as the plurality of biological images, by performing imaging while switching a wavelength range of light that forms an image on an imaging element included in the medical observation apparatus.
 15. The medical information processing apparatus according to claim 1, further comprising: a medical observation apparatus that is a medical endoscope or a medical microscope.
 16. A medical information processing apparatus comprising: processing circuitry configured to, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, select at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein at least a first captured image of the living body in a visible light region and a second captured image of the living body in a near-infrared region are included in the plurality of biological images, and wherein the processing circuitry is configured to select the second captured image of the living body in the near-infrared region as the recommended image in a case that the surgical characteristics include that smoke or mist is detected or a medical energy device is in use.
 17. The medical information processing apparatus according to claim 16, wherein the processing circuitry is configured to instruct a display screen to be switched from the recommended image to the first captured image of the living body in the visible light region in a case that the surgical characteristics include that no smoke or mist is detected and that use of the medical energy device is finished.
 18. A medical information processing apparatus comprising: processing circuitry configured to, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, select at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein the processing circuitry is configured to select, as the recommended image, an image of the secondary images in which a distribution of oxygen saturation generated as secondary information, based on the plurality of biological images, is visualized, or a captured image of the living body in a near-infrared region of the plurality of biological images, that corresponds to a distribution of blood, in a case that the surgical characteristics include that a blood flow obstruction medical instrument or a blood vessel anastomosis medical instrument, is detected.
 19. A medical information processing apparatus comprising: processing circuitry configured to, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, select at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein on the processing circuitry is configured to select a captured image of the living body of the plurality of biological images having a wavelength range permitting visual identification of a blood vessel, as the recommended image in a case that the surgical characteristics include that an existence of a blood vessel is detected.
 20. A medical information processing apparatus comprising: processing circuitry configured to, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, select at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image; and a medical observation apparatus that includes a multispectral sensor including at least a pixel configured to detect intensities of light in a plurality of wavelength ranges at one time while distinguishing intensities for each wavelength range, wherein a biological image for each of the plurality of wavelength ranges is generated from an output result of the multispectral sensor as the plurality of biological images.
 21. A medical information processing method comprising: selecting, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein the plurality of biological images include a first captured image of the living body in a visible light region and a second captured image of the living body in a near-infrared region; and changing a color tone of an area of the second captured image in which light, in the near-infrared region, has been detected to a predetermined color tone in the display.
 22. A non-transitory computer readable medium having stored therein a program that when executed by a computer causes the computer to execute a medical information processing method, comprising: selecting, based on surgical situation information concerning surgical characteristics at a time of observing an interior of a living body, at least one of a plurality of biological images each having a different wavelength range or at least one of secondary images generated from the plurality of biological images each having a different wavelength range, as a recommended image, wherein the plurality of biological images include a first captured image of the living body in a visible light region and a second captured image of the living body in a near-infrared region; and changing a color tone of an area of the second captured image in which light, in the near-infrared region, has been detected to a predetermined color tone in the display.
 23. A medical observation system comprising: a medical observation apparatus including first processing circuitry configured to generate a plurality of biological images by capturing images of an interior and exterior of a living body in a plurality of wavelength ranges; and an information processing apparatus including second processing circuitry configured to, based on surgical situation information concerning surgical characteristics at a time of observing the interior of the living body, select at least one of the plurality of biological images having different wavelength ranges or at least one of secondary images generated from the plurality of biological images, as a recommended image, wherein the plurality of biological images include a first captured image of the living body in a visible light region and a second captured image of the living body in a near-infrared region, and wherein the second processing circuitry is further configured to change a color tone of an area of the second captured image in which light, in the near-infrared region, has been detected to a predetermined color tone in the display. 